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ABSTRACT 


An  experimental  verification  of  the  continuous  adjustment 
procedure  considered  by  Sakrison  is  undertaken  in  this 
thesis.   The  adjustment  procedure  permits  design  of  a 
filter  system  by  continuous  adjustment  of  k  system  para- 
meters so  that  the  average  error,  weighted  by  a  convex 
error  criterion,  is  minimized.   For  experimental  verifi- 
cation on  typical  analog  equipment,  this  procedure  is 
applied  to  the  simple,  yet  typical,  problem  of  separating 
a  specified  "voice"  signal  from  broadband  noise.   The 
e^erimental  work  to  determine  the  time  required  for 
convergence  to  a  near  optimum  setting,  the  sensitivity  of 
the  system  to  the  initial  pareuaeter  settings,  and  the 
sensitivity  of  the  system  to  the  choice  of  parameters  of 
the  adjustment  procedure  showed  that  the  adjustment  proce- 
dure can  be  carried  out  on  existing  analog  equipmen-c. 
Further,  since  convergence  times  of  well  under  30  seconds 
can  be  obtained  with  this  system  and  the  system  does 
perform  satisfactorily  over  the  required  30  db  range  in 
noise  power  density,  it  is  applicable  to  use  in  an  adaptive 
filter  system  for  radio  communications. 
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CHAPTER  I 
INTRODUCTION 
A.   INTRODUCTION 

An  experimental  verif icatiojci  of  the  continuous  adjust- 
ment  procedure  considered  by  Sakrison[l]    is  undertaken 
in  this  thesis.  The  adjustment  procedure  permits  design 
of  a  filter  system  by  continuous  adjustment  of  k  system 
parameters  so  that  the  average  error,  weighted  by  a  convex 
error  criterion,  is  minimized.   For  experimental  verifica- 
tion on  typical  analog  equipment  this  procedure  is  applied 
to  the  simple,  yet,  typical  problem  of  separating  a  speci- 
fied "voice"  signal  from  broadband  noise.   The  separation 
of  the  desired  signal  from  an  input  consisting  of  the 
desired  signal  plus  noise  is  accomplished  by  linearly 
combining  the  outputs  of  k  low  pass  filters  for  minimum 
weighted  error.   The  experimental  procedure  is  to  determine 
the  time  required  for  convergence,  the  sensitivity  of  the 
system  to  the  initial  parameter  settings,  and  the  sensitiv- 
ity of  the  system  to  the  choice  of  parameters  of  the 
adjustment  procedure. 

*  Numbers  in  brackets  refer  to  items  in  the  Bibliography. 

#  Hereafter  referred  to  as  "the  adjustment  procedure" . 
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B.   BACKGROUND 

In  order  for  the  adjustment  procedure  to  be  applicable, 
the  error  criteripn  must  be  convex;  and,  other  than  requir- 
ing that  they  be  from  related,  bounded  ergodic  processes, 
no  restriction  is  placed  on  the  input  and  output .   The  form 
of  the  filter  considered  here  is  shown  in  Figure  I-l.   It 
consists  of  k  bounded,  time  invariant  operations  on  the 
input.   The  outputs  of  these  operations  are  to  be  linearly 
combined  in  such  a  manner  as  to  optimize  the  system  per- 
formance coefficients.   If  we  let  f .  ^  denote  the  output 

i,t 

of  the  i   such  operation,  f .    must  have  a  correlation 

i,t 

with  any  linear  combination  of  the  other  f .  ^  which  has  a 
magnitude  less  than  one. 

A  block  diagram  of  the  basic  system  is  shown  in  Figure 
1-2  for  a  system  with  two  parameters,  i.e.  k  =  2.   This 
system  obtains  its  convergence  by  approximating  the  gradi- 
ent of  the  regression  surface,  M(x2^,x^)  =  E^(  e(t)  )j, 
being  searched  and  adjusting  the  values  of  xj^  and  X2  to 
move  the  system  toward  the  minimum  error.   Through  the 
function  c(t)  a  known  plus  and  minus  pertubation  to  the 
present  setting  of  x  and  x  is  introduced.   When  the  errors 

•J.  ^ 

th 
resulting  from  these  plus  and  minus  pertubations  of  the  i 

parameter  are  measured,  weighted  by  the  chosen  error 


Filter 
input 


v(t) 


CONTINUOUS 

BOUNDED 

OPERATIONS 

ON  V  (t) 

T<  t 

(LINEAR  OR 
NONLINEAR) 
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Figure    I-l.      Form  of  the  filter  (or  predictor  or  model)  to  be  designed.* 
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Figure    1—2.       Diagram  of  the  continuous  adjustment  procedure  for  two 
parameters,   x.  and  x^-    * 

Both  of  these  figures  taken  from  j^lj  . 
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criterion,  and  subtracted,  the  resultant  function, 
Y^(t)/c(t),  is  a  random  variable  whose  mean  is  a  differ- 
ence  approximation  of  the  i   component  of  the  gradient  in 
the  direction  of  the  optimum.   The  function  a(t)  is  a  mono- 
tonically  decreasing  function  which  is  then  used  to  weight 
the  value  of  the  gradient.   This  weighting  function,  a(t), 
approaches  zero  slowly  enough  that  sufficient  data  is 
averaged  to  achieve  the  statistical  optimum,  yet  approaches 
zero  rapidly  enough  that  the  setting  settles  to  a  fixed 
value.   The  change  of  the  parameter  is  then  given  by 


X,  (t)  =  X,  (1)  -  J  77^  Y.(y)dy. 


/ 


t 
c(y) 


The  form  a(t)  =  K^/Cat  +  bf^  and  c(t)  =  K^/(ct  +  d) 
satisfy  the  mathematical  requirements  for  convergence  if 
1/2<'=^^1  and  tf^>(l-c^)/4.   When  ex  =  1,  we  also  require 
that  O  <a/K3  <  4K  .   Here  K^  is  the  largest  number  such 
that  the  regression  surface  gradient  in  the  direction  of 
the  minimum  is  always  greater  than  K  times  the  distance 
to  the  minimum.   The  choice  CY  =  1  and  1^  ^   1/4  gives  a 
rate  of  convergence  which  cannot  be  exceeded  in  general 
for  all  processes  and  error  functions  meeting  the  previously 
stated  requirements .   The  initial  convergence  depends 
critically  on  the  adjustment  coefficients  a/K  »  c/K  ,  cy  , 
and  <^  . 
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A  particular  advantage  of  the  adjustment  procedure  is 
that  it  is  not  restricted  to  the  mean-square  error  criterion, 
and  it  can  be  carried  out  on  analog  equipment.   In  addition, 
if  one  desires  to  design  analytically  an  optimum  linear 
system,  it  is  necessary  to  first  measure  the  input  and 
output  statistics.   After  this  either  the  Weiner-Hopf 
equation  must  be  solved  or,  in  the  finite  parcumeter  case, 
a  matrix  must  be  inverted.   An  advantage  offered  by  this 
adjustment  procedure  is  that  these  two  steps  are  replaced 
by  a  single,  easily  mechanized  procedure. 

In  a  system  where  some  error  criterion  other  than  the 
mean-sqpaare  error  is  of  interest,  this  method  offers  the 
advantage  of  being  able  to  handle  the  broad  class  of  convex 
error  criterion.   This  latter  advantage  disappears  when  the 
input  and  output  a^re  jointly  gauss ian  since  for  this  case 
the  optimum  filter  for  the  mean-square  criterion  is 
the  optimum  filter  for  any  convex  error  criterion  C2j  - 
For  the  case  in  which  the  signals  involved  are  non-gaussian 
and  a  non-linear  filter  is  desired,  the  adjustment  proce- 
dure is  again  suitable.   For  the  situation  in  which  the 
noise  is  quasi-stationary  (nearly  constant  over  long  periods 
with  respect  to  the  time  required  for  the  procedure'  to 
settle  to  a  near  pptimum  setting) ,  this  procedure  would 
also  be  useful  in  an  adaptive  coi^figuration. 
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It  is  to  be  noted  that  this  system  requires  that  the 
desired  output  signal  be  available.   In  this  adjustment 
procedure  duplication  of  the  conditions  under  which  the 
filter  will  have  to  operate  (including  real  time  data  in 
the  operational  environment)  spares  one  the  necessity  of 
having  to  measure  the  statistical  character  of  the  input 
and  output.   In  a  design  based  on  mathematical  calculation 
of  the  optimum  coefficients  for  which  the  statistics  are 
unknown,  we  must  have  available  the  desired  output  for  a 
fairly  long  period  of  time  in  order  to  measure  the  appro- 
priate statistics.   Periodic  application  of  the  adjustment 
procedure  to  correct  for  slowly  changing  statistics 
requires  that  the  desired  output  be  available  at  the 
receiver  for  only  a  small  percentage  of  the  time.   More- 
over, in  certain  situations  of  interest  the  procedure 
can  be  carried  out  without  the  desired  signal  RlJ  .   It 
is  sufficient  to  know  that  the  previously  mentioned 
constraints  are  met. 

C.   TWO  ADAPTIVE  CONFIGURATIONS 

There  are  two  particular  problems  which  are  of  interest 
to  mobile  military  units  for  which  this  procedure  might 
provide  a  suitable  solution  in  an  adaptive  configuration. 
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Let  us  consider  the  problem  of  radio  communications  in 
the  presence  of  atmospheric  noise.   Crichlow  et  al.  [Js^] 
have  published  data  on  actual  radio  noise  measurements  at 
several  locations.   For  a  location  near  Front  Royal,  Va. 
at  a  frequency  of  2.18  Mc.  it  was  found  that  the  four  hour 
median  of  the  noise  level  varied  over  a  range  of  about  30 
db  during  a  mean  24  hour  period.   Deviations  of  up  to  10  db 
of  the  one  hour  mean  from  the  four  hour  mean  were  also 
recorded.   However,  it  was  found  that  when  the  short  time 
variations  in  noise  power  are  averaged  over  a  long  period 
(several  minutes)  the  average  noise  power  level  is  nearly 
constant  for  any  given  hour  with  variations  seldom  more 
than  2  or  3  db  except  during  sunrise  and  sunset  periods. 
Since  the  noise  is  quasi-stationary  for  a  period  of  about 
an  hour,  it  would  be  appropriate  to  apply  the  adjustment 
procedure  in  an  adaptive  fashion  in  a  radio  communication 
link  if  the  procedure  leads  to  convergence  within  a  period 
of  less  than  30  seconds  (convergence  times  much  less  than 
this  have  been  obtained  in  the  course  of  the  present 
investigation) .   Such  an  adaptive  system  could  be  used  to 
achieve  either  a  fixed  signal  to  noise  ratio  while  varying 
the  transmitter  power  or  a  maximum  signal  to  noise  ratio 
with  fixed  transmitter  power. 
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A  primary  consideration  for  operation  of  some  military 
communications  systems  is  the  concealment  of  the  location 
of  the  transmitting  unit  from  an  enemy.   With  a  capability 
of  precisely  adjusting  the  transmitter  power  for  a  desired 
signal  to  noise  ratio  at  the  receiver  (to  be  accomplished 
by  a  feedback  signal  to  the  transmitting  unit  from  the 
receiving  unit  whose  location  need  not  be  concealed)  we 
reduce  the  probability  of  enemy  location  of  the  transmit- 
ting site  compared  with  a  situation  in  which  the  same 
transmitter  power  is  used  for  all  noise  conditions. 

In  the  case  of  filtering  an  audio  signal  from  varying 
noise,  the  use  of  statistical  methods  are  possibly  of 
marginal  value  since  they  result  in  a  rather  modest  improve- 
ment over  filters  designed  by  intuitive  methods  (this  point 
will  be  discussed  later) ,   This  example  was  chosen  for  the 
present  work,  however,  because  it  is  fully  adequate  for  the 
purpose  of  evaluating  the  adjustment  procedure  and  could 
be  easily  implemented  with  available  equipment. 

One  example  of  interest  for  military  application  in 
which  substantial  improvement  is  possible  is  in  the  field 
of  magnetic  detection  devices.   In  particular,  we  might  be 
interested  in  detecting  submarines  by  aircraft  using 
magnetic  airborne  detection  (MAD)  equipment.   The  problem 
here  is  that  the  background  magnetic  signature  of  the  earth 
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changes  with  time  as  well  as  with  geographic  location.   In 
such  cases  an  aircraft  might  easily  encounter  several 
different  backgrounds  on  one  search  flight.   Different 
areas  might  be  searched  on  different  days,  also  causing  a 
change  in  the  background  noise  spectrum.   If  we  assume  an 
aircraft  speed  of  200  knots,  we  might  anticipate  that  the 
signature  of  a  submarine  would  last  for  no  longer  than  five 
or  six  seconds  as  a  pulse  introduced  into  the  background. 
Since  this  time  is  very  short  compared  with  the  time  over 
which  the  background  noise  remains  approximately  constant 
(an  hour  or  so) ^  the  adjustment  procedure  could  be  efficient- 
ly applied  here. 

If  one  conducts  experiments  with  friendly  submarines, 
it  is  possible  to  predict  the  pulse  shape  to  some  accuracy. 
Thus,  it  will  be  possible  to  have  available  a  desired 
signal  for  use  in  adapting  the  system  to  the  needs  at  hand. 
A  matched  filter  arrangement  can  be  improved  if  there  is 
a  means  of  adjusting  the  filter  to  varying  noise  spectra, 
which  is  the  case  considered  here. 

D.   A  PROPOSED  ADAPTIVE  SYSTEM 

For  purposes  of  checking  the  convergence  properties  of 
the  adjustment  procedure,  it  was  decided  to  employ  the 
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procedure  in  a  self  optimizing  or  adaptive  filter  for 
separating  a  speech  signal  from  broadband  noise  of  slowly 
varying  level.   Specifically,  an  adaptive  system  which 
might  have  application  in  a  radio  communication  network 
between  ships  was  selected. 

Pant  Q4II  describes  the  long-term  average  of  speech 
as  being  characterized  by  a  12  db  per  octave  decay  over  the 
audio  spectrum.  He  also  shows  scxne  curves  of  the  actual 
spectrum  which  suggest  that  restricting  the  circuit  usage 
to  certain  types  of  messages  (i.e.  a  tactical  voice 
circuit)  would  result  in  a  power  density  spectrum  which 
would  follow  the  12  db  curve  on  the  average,  but  which 
would  have  significant  plus  and  minus  perturbations  from 
this  mean.  With  such  a  desired  output  spectriim,  which  is 
essentially  a  staircase  function  with  steps  of  uneven 
height  following  the  mean  of  12  db  per  octave  slope  in  the 
frequency  domain,  it  was  decided  that  an  investigation 
should  be  made  using  an  adaptive  filter  system  consisting 
of  k  low  pass  filters  spaced  linearly  throughout  the  useful 
portion  of  the  audio  band  (250  to  2500  cps  contains  most 
of  the  useful  intelligibility) .   The  adaptive  system  would 
then  compare  the  desired  output  with  that  actually  being 
received  and  optimize  the  coefficients  of  the  k  filters 
for  minimum  mean-square  error. 
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In  order  to  show  that  an  adaptive  system  has  some 
merit  in  the  use  outlined  above,  let  us  examine  the  improve- 
ment in  signal-to-noise  ratio  of  the  adaptive  filter  over 
a  fixed  filter.   We  will  consider  that  we  have  available 
ideal  bandpass  filters  with  adjustable  multiplicative 
coefficients  (k. ) .   With  N  such  filters  linearly  spaced 
over  that  portion  of  the  audio  band  containing  the  useful 
intelligibility,  the  transfer  function  of  such  a  filter  is 

where  U_-.  (x-a)  is  a  unit  step  function  which  is  0  for 

X  <  a  and   1   for  x  >  a      and  co     =   2ir(250  +   2250ij  ^    i  =   o,    1, 

-■-  N 

2,...,N.   Note  that  N  ideal  bandpass  filters  can  be  obtained 
from  a  linear  combination  of  N  +  1   ideal  low  pass  filters. 

We  then  consider  the  spectrum  of  the  desired  signal 
as  being  of  the  form 

M 

J-/ 

where  6^  .  =  co   for  i  =  j  and  N  =  M  and  the  a .  are  per- 
D      i  D 

turbations  from  the  form  -,-- _____^__  for  the  a^ 

(3  CO  +  628)^  3 

evaluated  at  CJ>  •,    and  we  assume  that  the  a-  are  constant 

2^2250 

over  the   increment    A<sj     =     — -        .      We  assxame  that   the 

M 

signal  is  subjected  to  additive  noise  of  the  same  form  as 
the  signal  with  N  .  replacing  the  a . .   We  also  assxime  that 
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the  N  .  are  constant  across  the  increment  Loj  and  that  the 
cross  correlation  between  the  desired  signal  and  the  noise 
is  zero. 

With  these  assumptions  the  form  of  the  mean-square 
error  becomes 


nf'^  ^/"po 


~^=  -i-     \S    ri-H(s)J       Ql-HC-s)]    ds   +  -i-    JN 
2"3   y  2Trj    _^^ 

where 


H(s) 


2 


ds 


s  =  E^  [u_i(N-  6J.)  -  u_^(|(i;|-  o).   i)] 
J'  I 

(s)  =  2]  ^i  C^-idM-  c^i)   -  u_3^(|a:|-   oD  ._i3 


H. 

N  =X]^oj&-l^'^|-   ^  j^    -  U-iH^I-   CO       ^)] 
If  we   let  M  =  N  for  simplicity,    the  mean-square   error  is 

w        2_ I    1^        1  i      OLJ 

/=/ 

Then  by  taking  partial  derivatives  we  find  the  optimum  values 

^i  -T 

of  k,  ,   k.   _4.  =    , ,    ,  and  the  minimum  value  of  e^, 
i'    1  opt   ai+Noi 

jV  _,  A/ 


aiNc 


mm  n      Zja.+N   ,  ^       Zj     oi   i   opt    . 


As  a  check  note  that  when  N  •  =  0,  k.  =  1  and  when  N  .  =°^ , 

kj^  =  0  for  all  i. 

If  we  make  a  stepwise  approximation  to  the  mean  speech 

power  spectrum  described  by  Fant  \j^   with  M  =  N  =  10  and 

A  CO  =   2^2250,   we  have 
— W^ 
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a^  =  27.0         ac;  =  3.31        a©  =  1.26 


a2  =  12.6         ag  =  2.46        a^Q  =  1.00 
a^  =   7.08        a^  =  1.82 
a^   =  4.74        ag  =  1.50 

We  now  find  the  optimum  kj_  for  N^-  =  5,0  for  all  i.   We  now 

consider  that  these  Tc-    ^  are  those  which  would  have  been 

1  opt 

obtained  if  this  particular  type  filter  had  been  designed 
for  the  average  condition.   We  further  assxime  that  the 
actual  a^^  of  the  signal  are 

ai  =  30  ag  =  4  ag  =  1 

32  =  13  ag  =  2  a^o  =  ^ 

33  =  4  ay  =  2 

34  =  4  33  =  1 

These  values  are  chosen  to  obtain  about  the  same  input  signal 
to  noise  ratio  as  was  avail3ble  to  the  mesn  design.   The 
perform3nce  of  3  fixed  filter  comp3red  with  the  optimum 
filter  is  shown  in  T3ble  I. 

TABLE  I 
COMPARISON  OF  THE  OPTIMUM  FILTER  WITH  A  FIXED  FILTER 
^o    ^min   ®2fixed    %   improve   snr^^   ^nr^^^  sni^/snr^ 


•^  4.11  9.94  142.0        12.4  15.81  1.28 

5.0        19.62  20.2  2.8  1.24  2.96  2.39 

50.0        46.90  122.8  162.0  .124  .475  3.83 


Thus,  we  see  th3t  signific3nt  improvements  are  possible  with 
an  adaptive  system. 


20 


Having  seen  that  for  the  case  of  the  ideal  bandpass 
filter      it  is  possible  to  obtain  improvement  over  a 
fixed  system,  we  next  turn  to  the  problem  of  actually 
checking  the  convergence  of  such  a  system.   Considerations 
to  be  dealt  with  below  restrict  the  error  criterion  to 
either  an  absolute  value  or  a  mean-square  criterion.   Since 
the  mean-square  criterion  is  more  amenable  to  analytic 
treatment,  we  have  in  this  case  some  way  of  knowing  if  the 
results  given  by  the  system  are  near  the  optimum.   Hence, 
the  mean-square  error  criterion  was  chosen. 

The  spectrum  of  the  desired  voice  signal  is  of  second 
order.   In  order  to  obtain  sharp  resolution  in  our  process 
we  must  then  use  filters  of  at  least  third  order.   To 
reduce  the  analytic  treatment  to  reasonable  proportions 
and  to  use  the  tables  available  in  Newton,  Gould,  and  Kaiser 
[jsj   to  evaluate  the  error,  third  order  filters  were  chosen. 
These  tables  provide  an  easy  means  of  evaluating  the  mean- 
square  error  as  a  function  of  the  variable  parameters,  "k^. 
Having  the  mean-square  error  in  terms  of  the  k.  permits  us 
to  find  the  optimum  simply  by  solving  the  set  of  equations 
resulting  from  setting  the  partial  derivatives  with  respect 
to  each  of  the  kj^  equal  to  zero.   It  is  then  possible  to 
calculate  the  minimum  mean-square  error  for  comparison  with 
that  found  experimentally. 

21 


CHAPTER  II 
PROCEDURE 

A.   THE  ANALOG  COMPUTER 

One  of  the  advantages  claimed  by  Sakrison  for  his 
adjustment  procedure  is  that  it  can  be  realized  on  simple 
analog  computing  facilities  [ij  .   A  Philbrick  analog 
computer  was  made  available  by  the  Dynamic  Analysis  and 
Control  Labratory  so  that  this  procedure  might  be  checked. 
This  computer  consists  of  20  universal  linear  operator 
units  (K5U) ,  4  stabilized  multiply /divide  units  (K5M) ,  6 
unstabilized  multiply /divide  units  (MU/DV) ,    and  an  eight 
channel  oscilloscope  for  output  read-out. 

The  K5U  units  permit  operations  of  the  form 

4- 

^out^t>  =  ^o  -^  lO^'I^^i^ini^t)        -11.10  ^  ai  ^11.10 

<-  I 
or  m  =  0,  1,  2,  3 

^out^t)  =  e^  +  10"J'^aiein.(t)dt     -50.0  ^  e^  ^50.0 

These  units  have  a  frequency  response  which  is  flat  to 

beyond  6000  cps. 
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The  basic  equation  for  the  operation  of  the  K5M  units 

ei(t)e2(t) 
is   e   .  (t)  =  ,   These  K5M  units  also  permit  the 

out  -»   /4-\ 

e3(t) 
addition  of  a  constant  to  one  of  the  two  multiplication 
inputs,  to  the  division  input,  or  to  the  output.   Figure 
II-l  shows   the  output  amplitude  response  versus  frequency 
for  the  K5M  units.   In  addition  it  should  be  noted  that 
the  waveform  was  badly  distorted  above  1500  cps. 

The  MU/DV  units  have  a  response  characteristic  similar 
to  that  of  the  K5M  units.   In  the  multiplication  mode 

i-1     •  ^     L    •        r-       .  •,  r-  /.  \        ©1  (t)eo(t)         _„ 

their  output  is  of  the  form   ©out  ^^'    ~  —^ — • — = — -  •   ^® 

25 

drift  in  these  units  is  of  such  a  magnitude  that  they  must 
be  checTced  at  least  once  an  hour  to  insure  that  they  meet 
the  accuracy  required  by  the  problem  at  hand. 

The  maximum  range  without  saturation  for  all  these 
units  is  i  50  volts.   There  is  an  inherent  noise  level  of 
about  0.035  volts  rms  in  the  multiplication  units,  and  a 
noise  level  of  less  than  1  mv,  in  the  K5U  units.   The  above 
noise  levels  are  referred  to  the  input  so  that  for  high 
gains  the  noise  is  also  amplified.   All  of  these  units  are 
capable  of  driving  up  to  four  other  units. 
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B.   EXPERIMENTAL  PROCEDURE 

Experimental  work  was  undertaken  to  insure  that  the 
adjustment  procedure  would  lead  to  system  parameter  con- 
vergence on  the  available  analog  equipment.   In  particular, 
it  was  desired  to  know  if  the  system  parameters  would 
converge  within  a  time  suitable  for  the  application  proposed 
and  if  the  adjustment  procedure  was  critically  dependent  on 
any  of  the  adjustment  parameters.   Sakrison  has  shown  that 
for  a  related  discrete  adjustment  process  the  settings  of 
the  parameters  of  the  adjustment  process  are  quite  impor- 
tant \_6]    ,   Certain  settings  of  the  adjustment  parameters 
in  the  discrete  case  led  to  violent  oscillations  for  a 
prolonged  period  of  time  before  settling  down,  and  for 
other  settings  the  behavior  was  quite  overdamped  and 
convergence  did  not  occur  within  a  reasonable  period  of 
time. 

The  use  of  a  mean-square  error  criterion  in  this 
experimental  work  has  been  mentioned  above.   The  analytical 
convenience  offered  by  this  choice  is  obvious;  however,  our 
freedom  of  choice  is  not  as  great  as  might  be  implied  since 
the  number  of  available  multipliers  also  restrict  the 
critera  which  may  be  considered.   Better  appreciation  of 
the  latter  restriction  is  gained  by  examining  the  schematic 
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of  three  convex  error  criteria  for  adjustment  of  only  two 
filter  gains. 

Figure  II-2  shows  the  schematic  diagram  for  the  abso- 
lute value  criterion.   In  this  particular  arrangement  we 
have  simplified  the  problem  by  assuming  that  a(t)  =  c(t). 
It  is  seen  from  this  figure  that  two  multipliers  are 
necessary  for  each  parameter  to  be  adjusted. 

The  arrangement  shown  in  Figure  II--3  is  for  the  square 
error  criterion.   It  will  be  noted  that  fewer  linear  oper- 
ator units  are  required  for  this  configuration  than  for  the 
absolute  value  arrangement,  but  that  the  number  of  multi- 
pliers remains  at  two  per  parameter  being  adjusted. 

In  order  to  meet  the  requirements  of  a  fourth  power 
error  criterion.  Figure  II-4,  eleven  multiplier^  or  4  per 
parameter  being  adjusted  plus  one  common  to  all  the  adjust- 
ment parameters,  are  needed.   For  the  analog  computer 
available  for  this  work  the  constraint  imposed  by  the  number 
of  multipliers  available  quickly  limits  us  to  either  the 
absolute  value  or  to  the  square  error  criterion.   The  choice 
of  the  latter  was  then  made  on  the  basis  of  computational 
convenience. 

It  was  decided  that  extensive  studies  of  the  perform- 
ance of  the  adjustment  procedure  for  a  filter  system 
consisting  of  two  filters  should  be  conducted.   On 
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Pig'ire  II-2.      Schenia'ric   diagram  for  absolute  value  error 
criterion. 
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Figure  II-3.   Schematic  diagram  for  square  error  criterion. 
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two  j: liter  sy stent. 
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completion  of  these  tests  examination  of  a  four  parameter 
should  be  conducted.   In  the  four  parameter  case,  analyti- 
cal determination  of  the  optimum  values  of  the  coefficients 
is  difficult  since  available  tables  do  not  permit  evalua- 
tion of  twelfth  and  fourteenth  order  integrals.   For 
purposes  of  this  experiment,  however,  it  was  sufficient 
to  show  consistent  values  of  the  mean-square  error  from  the 
four  parameter  system  optimized  by  the  adjustment  procedure. 

In  the  four  parameter  case  eight  K5U  units  are 
required  for  auxiliary  uses  associated  with  signal  shaping, 
function  generation,  and  summing  operations  common  to  all 
filters.   Since  one  integrator  (K5U)  is  required  for  each 
parameter  adjusted  and  simulation  of  third  order  filters  on 
the  analog  computer  requires  the  use  of  three  K5U  units  for 
each  filter,  there  are  an  insufficient  number  of  computing 
elements  available  for  the  four  parameter  case  entirely 
simulated  on  the  computer.   Therefore,  in  order  to  obtain 
consistent  results  between  the  two  and  the  four  parameter 
case,  filters  formed  by  RC  networks  in  the  input  and  feed- 
back paths  of  stabalized  operational  amplifiers  were  used. 
The  form  of  the  transfer  function  of  the  filters  is 

H(s)  = .   The  time  constants  of  the  filters 

(a]^s+l)2(a2S+l) 

used  in  the  experimental  work  are  shown  in  Table  II. 
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Filters  2  and  3  were  used  for  the  two  filter  studies.   The 
weightings  of  their  outputs  are  the  parameters  to  be  opti- 
mized, and  these  are  hereafter  called  k^^  and  k-  for  filters 
2  and  3  respectively. 

TABLE  II 
TIME  CONSTANTS  OP  FILTERS  USED  FOR  EXPERIMENTAL  WORK 


Filter  number 


ao 


32 


1 

.239 

.00130 

.00635 

2 

.187 

.000424 

.000213 

3 

.185 

.000252 

.000130 

4 

.140 

.000180 

.0000916 

Figure  II-5  shows  the  computer  prograunci  for  the  experi- 
mental work  with  two  filters.   This  program  follows  the 
diagram  shown  in  Figure  II-3  except  for  the  zunplifiers 
following  the  filters.   These  amplifiers  are  necessary, 
in  this  case,  to  increase  the  level  of  the  waveform  at 
that  point  to  avoid  computer  noise.   In  Figure  II-5 
capital  letters  are  gains  of  the  associated  K5U  units, 
underlined  capital  letters  are  initial  conditions,  and 

small  letters  denote  the  time  function  at  the  output  of  the 

CRki  ' 
associated  unit.   From  Figure  II-5  it  is  seen  that  k^  =  l  , 

RUk  '       " A    5 Oe^ ' 
k2  =    2  ,  and  e^  s=  ,  where  the  kj^  are  the  parameters 

25  E 


being  optimized  for  minimum  e^. 
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The  desired  output,  S,  is  obtained  by  passing  the  output 

of  a  white  noise  generator  through  a  filter  of  the  form 

K 

~j-. -  ■.  2~   .   This  gives  the  desired  approximation  of 

{,"}  CO    +d2o) 

a  mean  speech  spectrum. 

To  the  desired  output  was  added  white  noise  from 
another  generator.   A  separate  noise  generator  was  used  to 
insure  minimum  correlation  between  the  desired  output  and 
the  noise.   This  sum,  when  used  as  the  input  to  the  filter 
system,  gave  a  good  approximation  to  speech  distorted  by 
the  type  of  noise  likely  from  atmospheric  sources. 

The  values  of  both  the  signal  power  density,  Sq,  and 

the  noise  power  density,  N^,  were  determined  by  assuming 

that  they  were  from  white  noise  sources  having  a  flat 

power  density  spectrum  beyond  the  maximum  frequency  of 

interest.  (Later  measurements  of  the  power  density  spectra 

of  the  two  sources  validated  this  assumption.)  For  the 

purpose  of  measuring  N  ,  the  output  from  the  white  noise 

generator  was  passed  through  a  single  pole  low  pass  filter 

having  a  half-rpower  cut-off  freqiaency  about  equal  to  the 

highest  value  to  be  used  in  any  of  the  test  filters.   The 

resultant  waveform  was  squared  and  averaged  by  integrating 

the  squared  output  for  one  second.   The  tables  of  Newton, 

Gould  and  Kaiser  L5j  were  then  used  to  determine  the  value 

of  N  . 
o 
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The  procedure  used  to  evaluate  S  was  the  same  as 
that  above  except  that  the  white  noise  was  shaped  prior 
to  being  inserted  into  the  single  pole  low  pass  filter. 
The  same  low  pass  filter  was  used  for  both  cases. 

The  function  a(t)  was  generated  by  inserting  a  ramp 
of  the  form  A't  +  2  into  the  division  input  of  a  K5M  unit 
and  then  multiplying  the  output  of  the  K5M  unit  by  the  inte- 
grator gain,  D,  so  that  the  form  of  a(t)  was  -, — ,   ».  .   It 
was  found  that  constants  less  than  2  in  the  ramp  function 
caused  internal  saturation  in  the  K5M  units  and  that 
inconsistent  results  were  obtained  when  such  saturation 
occurred.   Since  the  lowest  constant  value  was  desired  to 
derive  maximxim  benefit  from  a(t)  as  a  time  function,  the 
value  2  was  chosen. 

The  experimental  value  of  the  mean-square  error  was 
found  by  squaring  the  error  signal  and  integrating  this 
value  for  a  fixed  time  period.   Time  periods  of  0.1,  1, 
and  10  seconds  were  used  depending  on  the  rate  of  change 
of  the  error. 

For  purposes  of  this  study  convergence  time  is  defined 
as  the  time  elapsed  from  activation  of  the  system  until  the 
last  time  that  the  mean-square  error  falls  to  within  10% 
of  its  minimum  value.   That  region  containing  all  those 
points  having  their  mean-square  error  within  10%  of  the 
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minimum  value  is  hereafter  called  the  convergence  area. 
Although  this  definition  of  convergence  is  quite  arbitrary, 
it  does  give  a  fairly  concise  definition  of  the  time  taken 
to  reach  near  optimum  performance.   A  value  less  than  10% 
would  lead  to  a  greater  variance  in  the  convergence  time 
due  to  the  decreased  gradient  of  the  regression  surface 
in  the  vicinity  of  the  optimum. 

Convergence  times  were  obtained  by  stop  watch  measure- 
ments while  observing  an  oscilloscope  displaying  k,  versus 
k^  and  noting  the  time  to  enter  the  convergence  area  which 
had  been  drawn  on  the  face  of  the  CRT. 

As  a  matter  of  convenience,  the  term  "feedback  loop" 
is  defined  as  referring  to  that  portion  of  the  system  from 
the  output  of  the  error  summing  K5U,  e(t)  in  Figure  II- 5, 
through  multiplier  3 (or  4)  and  integrator  5 (or  6)  to  multi- 
plier l(or  2) . 

The  results  desired  from  the  experimental  work  were: 
1)  the  effect  of  the  initial  settings  of  the  kj^  on  conver- 
gence time,  2)  the  effect  of  feedback  loop  integrator  gain, 
D,  on  convergence  time  with  A'  fixed  (note  that  the  same 
gain  setting  is  used  for  all  feedback  loop  integrators) , 
3)  the  effect  of  changes  of  A'  on  convergence  time  for 
D  fixed,  4)  the  range  of  noise  power  density  over  which 
it  is  possible  to  obtain  convergence  with  S^  fixed, 
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5)  the  values  to  which  the  k^  converge,  and  6)  any  other 
information  concerning  the  use  of  analog  equipment  to 
implement  this  adjustment  procedure. 

The  desired  form  of  the  experimental  data  was:  1) 
momerical  time  values  where  convergence  time  was  involved, 

2)  recordings  of  the  k^  and  e^  as  functions  of  time,  and 

3)  numerical  data  on  the  final  values  of  the  k- .   The 
recorded  data  were  taken  on  a  Sanborn  two  channel  recorder, 
Final  values  of  the  k^  were  recorded  at  the  end  of  50 
seconds  for  A'  =1, 
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CHAPTER  III 
RESULTS 

A.   THEORETICAL  RESULTS 

In  order  to  check  on  the  accuracy  of  the  convergence 
procedure,  studies  were  made  of  the  theoretical  values  of 
the  optimum  k.  and  the  minimum  mean-square  error  for  the 
two  filter  case.   The  problem  to  be  solved  was  in  the  form 

^=   J   ^  .,  ^^     Cl-H(s)]   [l-H(-s)]      ds 

--^  w2s'^+2Ws+l  W'^s^-2Ws+l 


+   \  N  H(s)H(-s)ds  (1) 


-r 

where      W  = 


628 


H(s)  =     ^1^0,2  +     ^2^0.3  

(a^  2^+1)  (^2, 2^"^ ■'•^      (ai^3S+l)'^(a2^3S+l) 


S  =  Power  density  spectrum  of  the  desired  signal 
before  shaping 

Nq  =  Power  density  spectrum  of  noise 

a.  .=  i    time  constant  of  j    filter (see  Table  II) 

We  see  that  a  sixth  and  an  eighth  order  integral  (  in 
the  Newton,  Gould,  and  Kaiser  sense  [sj  )  must  be  evaluated 
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to  obtain  the  desired  result.   Since  such  calculations  are 
exceedingly  long  and  many  similar  calculations  were  desired 
for  purposes  of  properly  evaluating  the  proposed  experi- 
mental work,  the  use  of  a  digital  computer  was  expedient. 
Time  was  made  available  by  the  Civil  Engineering  Systems 
Labratory  on  their  IBM  1620  computer  for  this  purpose. 

A  program  was  written  which  used  the  time  constants 
of  each  of  the  filters,  the  signal  power  density,  the 
noise  power  density,  the  experimental  values  of  the  k- , 
and  the  time  constant  of  the  desired  signal  shaping  filter 
to  compute  the  necessary  values  of  the  experimental  mean- 
square  error,  following  the  appropriate  tables  in  Newton, 
Gould,  and  Kaiser  L^J  .   It  can  be  shown  that  solution  of 
(1)  following  these  tables  leads  to  an  equation  of  the  form 

e^  =  CQ+C3^k^+C2k2+C3k^k2+C^kj^  +C5k2  (2) 

where  the  C^^  are  constants.  The  equations  necessary  for  the 
computation  of  the  optimum  values  of  the  k.  and  the  minimum 
mean-square  error  were  easily  obtained  from  (2) .   These 
equations  were  inserted  in  the  digital  computer  program  so 
that  for  any  given  set  of  values  of  signal  power  density, 
noise  power  density,  and  experimental  kj^,  it  is  possible 
to  compute  1)  the  theoretical  value  of  the  mean-square 
error  based  on  the  experimental  kj^,  2)  the  optimum  kj^,  and 
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3)  the  minimum  mean-square  error.   The  results  of  these 
theoretical  studies  are  presented  in  Figures  III-l  through 
III-7. 

Figure  III-l  shows  the  variation  of  the  minimum  value 
of  the  mean-square  errojr  with  noise  power  while  holding  the 

signal  power  density  fixed.   With  zero  noise  the  minim\im 

2 
mean-square  error  is  3.784  volt  ,  while  with  infinite 

noise  the  minimum  mean-square  error  is  90.275  volt^.   For 

_2 

Nq  =  8 . 56x10   the  noise  contributes  about  half  the  error 

in  the  output  of  the  optimum  system. 

Figure  III-2  demonstrates  the  range  of  the  "k^   with 
varying  noise  power  density  for  fixed  signal  power.   It  is 
to  be  noted  that  the  k.  go  to  zero  at  infinite  noise,  so 
that  the  maximum  possible  error  is  when  the  system  is 
completely  silent.   Such  a  range  in  the  values  of  the  kj^ 
with  changes  in  noise  power  density  indicates  that  an 
adaptive  system  might  be  worthwhile  even  for  this  simple 
case. 

If  we  pick  the  values  of  kj^  that  are  optimum  for  one 
noise  level  as  fixed  coefficients  of  our  filters,  it  is 
possible  to  demonstrate  the  effect  of  a  fixed  filter 
operating  on  an  input  of  fixed  signal  power  but  varying 
noise  power  density.   Such  a  comparison  is  presented  in 
Figure  III-3.   We  see  that  for  a  plot  of  the  mean-square 
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error  of  each  of  the  fixed  filters  versus  noise  power 
density,  the  mean-square  error  curve  is  tangent  to  the  curve 
of  the  minimum  error  at  the  point  for  which  the  k^^  of  the 
fixed  filter  are  optimum,  but  that  when  the  noise  changes 
from  the  value  for  which  the  k.  are  optimum,  the  fixed 
filter  error  curve  diverges  from  the  minimum  error  curve. 
It  should  be  noted  here  that  for  the  simple  case  being 
considered,  it  is  possible  to  obtain  excellent  results  by 
the  simple  expedient  of  building  a  switch  which  selects  one 
of  the  three  sets  of  coefficients.   While  such  a  procedure 
is  obviously  more  economical  than  an  adjustment  procedure 
in  this  particular  case,  it  must  be  remembered  that  this 
is  but  a  small  part  of  the  total  system,  and  that  addition 
of  more  filters  would  increase  the  number  of  possible 
combinations  of  the  k.  to  the  point  that  manual  switching 
would  be  out  of  the  question. 

For  the  case  studied  here,  the  regression  surface 
being  searched  is  a  paraboloid  when  mapped  using  k-^^,  k2» 
and  e^  in  a  right-handed  coordinate  system.   In  order  that 
the  reader  might  observe  the  effect  of  noise  power  density 
on  the  shape  of  this  surface,  three  graphical  displays  are 
made  here.   Figures  III-4  and  III-5  show  the  locus  of  points 
on  the  surface  at  planes  of  constant  mean -square  error  for 
N  =  8-56x10""^  and  8.56x10""^  respectively.   Figure  III-6 
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Figure  III-4.   e2  vs.  kj^  and  IC2  for  N^  =  8.56xl0~  ,  S^  =  0.575 
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-2 


Figure  III-5o   e2  vs.  ki  and  kj  for  N^  =  8.56x10"^,  S^  =  0.575 
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-2 
Figure  III-6.   Regression  surface  for  N^  =  8.56x10   ,  Sq  =  0.575 
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shows  the  three  dimensional  configuration  of  the  surface  in 

the  vicinity  of  the  minimum  for  N^  =  8.56x10""^.   For  both 

noise  levels  it  is  noted  that  the  planes  of  constant  e^ 

intersect  the  regression  surface  in  ellipses.     It  is 

possible  to  determine  the  equation  of  the  major  axis 

(hereafter  referred  to  as  the  line  of  minima) of  these 

ellipses  in  terms  of  the  k^  by  algebraic  manipulation  of 

(2) .   These  equations  are  found  to  be 

kj^  =  4.704  -  1.005  k2   for  N^  =  8.56x10-3   and 

ki  =  3.213  -  1.036  k2   for  N^  =  8.56xl0"2. 

All  of  the  theoretical  cases  considered  above  have 

been  for  S^  =  0.575  volt^sec.   The  variation  of  S^  from 

this  value  was  also  studied  for  N  =  8.56x10"^.   The  effect 

o 

on  minimum  mean-square  error  and  the  kj^  is  shown  in  Figure 
III-7.   We  see  that  th$  effect  on  the  error  in  this  region 
for  small  pertubations  of  S^  from  its  standard  value  is 
not  too  great. 

The  data  obtained  from  these  theoretical  studies  were 
used  to  define  the  convergence  area  in  terms  of  k]^  and  k2 
so  that  convergence  times  could  be  obtained  as  outlined  in 
Chapter  II.   The  computer  program  was  also  used  to  obtain 
the  value  of  e  based  on  the  k^  for  experimental  runs .   The 
results  of  the  digital  computer  calculations  are  presented 
in  this  section  as  well  as  in  the  next  section. 
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B.   EXPERIMENTAL  RESULTS 

A  study  of  the  feedback  loop  integrator  gain,  D,  as  it 
affects  the  convergence  time  is  presented  in  Figure  III-8. 
(For  this  figure  and  those  following  A'  =  1,  S  =  0.575, 
and  Nq  =  8.56x10   unless  otherwise  noted.)   For  values  of 
D  greater  than  10  the  system  failed  to  keep  the  error 
within  the  convergence  area  for  any  consistent  period  of 
time.   Further,  convergence  was  not  necessarily  obtained  at 
all.   For  runs  with  the  gain  less  than  10  it  was  found 
that  there  was  at  most  one  overshoot  of  the  convergence 
area  and  that  it  was  in  the  initial  phase  of  the  conver- 
gence.  On  entering  the  convergence  area  after  the  initial 
overshoot,  if  any,  it  was  found  that  the  error  remained 
within  the  convergence  area  throughout  the  remainder  of  the 
run. 

An  example  of  this  overshoot  is  shown  in  Figure  III-9. 
The  initial  settings  of  the  kj[  were  k-j^  =  k2  =  12  (hereafter 
the  notation  (ki,k2)  will  be  used)  with  D  =  1.   Note  that 
the  line  of  minima  was  reached  in  about  the  same  time  as  that 
taken  by  the  system  shown  above  starting  at  (6,6)  to  reach 
the  convergence  area . 

The  variation  of  mean  convergence  time  with  A'  for  D=l 
is  shown  in  Figure  III-IO.   For  all  of  these  runs  the 
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Figure  III-80   Convergence  time  versus  integrator  gain. 
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Figure  III-9o  Typical  convergence  of  the  k. (with  overshoot) 
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Figure  III-IO.   Convergence  time  versus  A'. 
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initial  values  were  (6,6).   The  path  followed  from  the  ini- 
tial point  to  entry  into  the  convergence  area  was  essentially 
the  same  in  all  cases  (i.e.  direct  to  the  convergence  area 
on  a  line  nearly  perpendicular  to  the  line  of  minima) .   In 
no  case  did  the  system  fail  to  converge. 

In  figure  III-ll  the  mean  convergence  time  is  shown 
for  several  initial  values  of  the  k • .   The  line  of  minima 
and  the  convergence  area  are  also  shown.   In  all  cases  the 
initial  movement  was  nearly  perpendicular  to  the  line  of 
minima.   Large  overshoot  of  the  convergence  area  occurred 
from  (-6,-6)  and  a  slight  overshoot  of  the  line  of  minima 
occurred  from  (-6,0) .   For  all  of  the  runs  shown  on  this 
figure  the  integrator  gain,  D,  was  two  since  this  value 
permitted  convergence  times  on  all  runs  within  48  seconds, 
the  maximum  time  allowed  if  a(t)  is  to  be  fully  utilized. 

A  demonstration  of  the  results  of  the  convergence 
procedure  over  a  30  db  range  of  noise  power  density  is 
shown  in  Figures  III-12  and  III-13.   S  =0.79  volt^sec 
was  used  for  the  runs  in  both  these  figures .   The  values 
of  kj^  and  k,  to  which  the  experimental  runs  converged  and 
the  theoretical  values  of  the  k^  over  the  30  db  range  of 
noise  power  density  are  shown  in  Figure  III"12o   The 
minimum  mean-square  error,  the  mean-square  error  calculated 
by  the  method  of  the  preceding  section  for  the  experimental 
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kj^  based  on  the  assumed  spectra,  and  the  experimentally 
measured  mean-square  error  are  shown  in  Figure  III-13. 
Some  saturation  was  encountered  in  the  input  K5U  for  the 
data  point  having  the  largest  noise  density,  but  this  could 
be  removed  by  linear  attenuation  of  the  input  signal. 

Figures  III-14  and  III-15  are  typical  recordings  show- 
ing the  convergence  of  the  k^  versus  time.   Values  of  the 
k.  taken  from  these  figures  demonstrate  the  movement  of  the 
error  to  the  line  of  minima  and  then  into  the  convergence 
area  when  plotted  as  in  Figure  III-9. 

Figures  III-16  through  III-24  show  the  mean-square 

error  and  the  parameter  k^(in  this  case  k    represents 

^  2 

the  coefficient  of  filter  2)  as  a  function  of  time  for  the 
four  filter  case.   In  order  to  demonstrate  both  the  range 
of  the  mean-square  error  and  the  fact  that  the  system  error 
converges  to  about  the  same  value  for  several  conditions 
of  internal  parameter  settings,  two  scales  are  used  on  the 
recordings  of  the  mean-square  error.   The  initial  squared 
error  is  integrated  from  -50  volts  for  0.1  seconds.   After 
the  error  has  reached  a  value  near  its  minimum,  the  squared 
error  is  amplified  by  a  factor  of  four  and  integrated  from 
0  volts  for  0.1  seconds.   The  apparent  discontinuity  in  the 
recording  indicates  the  change  of  the  initial  value  of  the 
integrator,  and  the  increased  amplitude  in  the  integrated 
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squared  error  a  short  time  later  indicates  the  introduction 
of  the  factor  of  four  cimplif ication.   The  results  of  several 
selected  runs  are  shown  in  Table  III. 


2 
2 


.5 


.5 


TABLE  III 


RESULTS  OF  SELECTED  RUNS  WITH  FOUR  FILTERS 


Initial  Value 


Final  Value 


ki '   ko'   k; 


lli    ]ii    lii 


Final  Mean-     Run 
Square  Error  Number 


.-2 


For  the  following  7  runs  Nq  =  8.56x10' 
30    30    30    30   -2    10    7    -2      28.5 


19    -2 


-30  -30  -30  -30 


-30   -30    30    30    -3    15 


34   -27 


-3 


30    30   -30   -30    -7    21   -19    13 


30    30    30    30   -10    36   -22 


14 


26.7 


32.0 


25.0 


25.0 


25.0 


30    30    30    30    0    10    4     3      25.0 

—  3 
For  the  following  7  runs  Nq  =  8.56x10 

30    30    30    30   -15    31     3 


0   -15 


30 


-30   -30   -30   -30   -12    10   42   -25 


-30   -30    30    30    -9 


19 


30    30   -30   -30   -21    48    -3 


30    30    30    30   -22    50   -16 


30    30    30    30 
NOTE:  Multiply  kj^^to  get  k^ 


-9    17 


4F1 
4F2 
4F3 
4F4 

4F5 
4F6 
4F7 


3 

9.9 

4F8 

0 

11.1 

4F9 

25 

9.9 

4F10 

6 

8.7 

4F11 

-4 

9.9 

4F12 

6 

11.1 

4F13 

9 

9.9 

4F14 
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It  was  found  that  about  45  minutes  were  required  to 
set  up  the  computer  for  each  set  of  runs.   The  largest 
portion  of  this  time  was  spent  in  adjusting  the  MU/DV  units 
for  the  proper  value  of  their  division  constant  and  removing 
the  mean  DC  level  from  them. 

Removal  of  the  average  DC  level  from  the  K5M  units 
was  also  necessary.   It  was  found  that  0x0/50  ^   0x0/20 
and  that  0x50/50  ^   0x50/20  ?^  0x20/50  for  operations  with 
the  K5M  units .   In  view  of  the  extreme  ranges  encountered 
in  conducting  the  tests  discussed  above,  this  was  particu- 
larly disconcerting.   The  values  involved  were  small,  on 
the  order  of  0.035  volts  rms,  but  when  these  are  integrated 
for  a  period  of  several  seconds,  they  begin  to  contribute 
an  error  about  which  little  can  be  done. 

C.   DISCUSSION  OF  RESULTS 

In  considering  the  effect  of  integrator  gain,  D,  on 
convergence  times  when  A'  =  1,  we  first  examine  both 
extremes  in  gain  settings,.   It  is  easily  seen  that  for  an 
integrator  gain  of  zero,  convergence  can  never  be  obtained, 
and  we  expect  that  small  values  of  gain  setting  will  lead 
to  slow  convergence.   At  high  gains,  the  50  volt  upper  limit 
of  the  computing  equipment  enters  the  problem  in  two  ways. 
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First,  a(t)  will  be  at  a  constant  value  (saturation)  for 
t  ^   D/50  -  2,  D  >:  100.   Second,  since  the  function  A't  +  2 
reaches  its  maximum  value  at  t  =  48  seconds  for  A'  =  1, 
the  minimum  value  which  a(t)  can  reach  is  D/50,   Both  of 
these  effects  violate  the  requirement  that  a(t)  decrease 
monotonically.   The  first  effect  means  that  the  convergence 
procedure  does  not  start  to  settle  down  until  some  time 
greater  than  zero,  and  the  second  effect  may  result  in 
undamped  oscillations  if  a  near  steady  condition  has  not 
been  reached  by  the  system  prior  to  the  time  a(t)  reaches 
its  minim\im  value.   In  any  event,  we  desire  that  a(t)  be  as 
near  zero  as  possible  for  t  greater  than  some  Tq  and  that 
there  be  no  initial  saturation.   Both  of  these  conditions 
are  met  for  small  values  of  A'.   In  general,  we  expect 
that  there  will  be  some  value  of  D  which  will  minimize 
the  time  to  convergence,  and  that  there  will  be  some  range 
of  settings  of  D  about  this  minimum  which  will  give  suitable 
values  of  convergence  time. 

From  Figure  III-8  it  is  seen  that  an  asymptotic  approach 
to  infinite  convergence  time  is  obtained  for  integrator 
gains  approaching  zero.  Minimum  convergence  time  is  obtain- 
ed for  an  integrator  gain  setting  of  two.   Convergence  time 
is  found  to  remain  nearly  constant  for  integrator  gains 
between  four  and  seven.   Further,  it  was  observed  that  for 
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values  of  gain  less  than  two,  the  system  converged  without 
any  overshoot  of  the  convergence  area.   However,  for  set- 
tings of  D  greater  than  two,  overshoot  became  increasingly 
severe.  Thus,  the  minimum  time  to  convergence  is  obtained 
for  a  setting  of  D  just  less  than  that  value  which  first 
results  in  oscillations. 

The  combined  effect  of  the  shape  of  the  regression 
surface  and  the  weighting  of  a(t)  shown  in  Figure  III-9 
is  perhaps  the  best  graphical  portrait  of  the  convergence 
process.   The  initial  movement  of  the  error  perpendicular 
to  the  line  of  minima  shown  here  is  typical.   There  is  an 
overshoot  of  the  convergence  area,  but  the  system  corrects 
itself  and  turns  back  toward  the  line  of  minima.   As  the 
error  decreases  while  following  the  line  of  minima  into 
the  convergence  area,  we  see  the  increasingly  slower  move- 
ment which  is  also  characteristic  of  the  system. 

For  the  experimental  system  studied  in  this  work  the 
convergence  time  is  less  than  one  second  for  A' ^  2.5. 
When  the  product  A't   (where  t^  =  convergence  time)  is  less 
than  0.2,  we  expect  this  product  will  be  dominated  by  the 
constant  2  in  A't  +  2,  and  that  there  will  be  no  further 
change  in  the  convergence  time  for  decreasing  values  of  A' . 
This  effect  is  shown  in  Figure  III-IO  where  the  curve  of 
convergence  time  versus  A'  has  a  flat  portion  for  A'  <  0.5. 
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For  each  combination  of  noise  power  density  and  signal 
power  density  there  is  a  fixed  value  of  K  .   Since  it  is 

required  that  0  ^  A'  ^  4Kq  when  D  =  1,  it  is  expected 
that  increasing  A'  beyond  A*  =  4Kq  will  result  in  increas- 
ing amounts  of  time  being  taken  for  the  system  to  converge. 
Although  K^  was  not  canputed  for  this  experimental  system, 
it  is  noted  that  for  10  >  A'  >  2  the  convergence  time 
increases  linearly  with  A'.   Since  oscillations  were  absent 
throughout  the  range  0  <  A'  <^  10,  we  may  conclude  that  the 
convergence  procedure  is  not  overly  dependent  on  A'  when 
convergence  occurs  prior  to  the  time  that  a(t)  reaches  its 
minimum  value. 

Prom  the  convergence  times  shown  in  Figure  III -11^  it 
is  seen  that  the  system  takes  very  little  time  to  reach  the 
line  of  minima  while  taking  the  majority  of  the  convergence 
time  to  move  down  the  valley  along  the  line  of  minima. 
For  instance,  if  we  look  at  the  two  points  (-6  6)  and 
(0,6),  we  see  that  the  convergence  time  is  about  the  same 
from  both  points^  but  that  a  slightly  longer  time  is  taken 
by  the  movement  from  the  initial  setting  furtherest  from 
the  line  of  minima.   Looking  at  the  two  points  (6,0)  and 
(6,-6)  we  see  that  the  perpendicular  distamce  to  the  line 
of  minima  is  about  the  same^but  that  for  the  initial  setting 
furtherest  from  the  convergence  area,  the  convergence  time 
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is  about  a  third  longer  compared  with  the  nearer  initial 
setting. 

The  seeming  deviation  from  this  analysis  is  present 
if  one  considers  the  points  (6,6)  and  (-6,-6).  Here  the 
discrepancy  is  due  to  an  overshoot  when  starting  from 
(-6,-6)  which  does  not  occur  when  starting  from  (6,6). 
After  the  overshoot  occurs  the  system  returns  to  the  line 
of  minima  at  about  (10,-8)  and  the  excessive  time  is  spent 
in  transit  down  the  line  of  minima  from  that  point.  After 
a  slight  overshoot  of  the  line  of  minima  from  the  initial 
position  of  (0,-6)  the  system  returns  to  the  line  of  minima 
only  a  short  distance  from  the  convergence  area.   From 
the  foregoing  discussion  we  again  see  that  the  system 
giving  the  most  rapid  convergence  results  when  the  gain 
settings  are  so  adjusted  that  the  system  is  just  on  the 
verge  of  overshooting  the  convergence  area.   Moreover^  the 
range  in  convergence  times  from  0.45  to  38.5  seconds  is 
not  indicative  of  the  best  possible  performance  of  the 
system  since  all  of  these  runs  were  obtained  with  that 
value  of  D  and  A'  which  gave  minimum  convergence  time  from 
(6,6). 
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The  information  shown  in  Figures  III-12  and  III-13 
demonstrate  that  this  system  is  capable  of  operating 
properly  over  a  range  of  30  db  of  noise.   Thus  the  system 
is  suitable  for  the  proposed  application  in  an    adaptive 
filter  for  radio  communications. 

Prom  Table  III  and  Figure  III-l  it  is  seen  that  in  the 
four  filter  case,  the  error  on  completion  of  optimization 
is  about  40%  less  than  that  obtainable  with  two  filters. 
This  is  to  be  expected  since  with  more  third  order  filters, 
we  can  more  closely  approximate  the  second  order  desired 
signal. 

A  computer  characteristic  which  may  have  introduced 
small  inaccuracies  in  the  experimental  results  was  the 
inability  to  adequately  compensate  for  the  input  dependent 
DC  level  of  the  multipliers .   Admittedly  this  level  was 
small,  but  it  can  become  quite  troubelsome  when  the  noise 
power  density  and/or  the  signal  power  density  changes  from 
one  trial  to  the  next  as  must  be  the  case  if  this  system 
is  to  be  useful  in  the  adaptive  configuration.   For  this 
work  we  could  adjust  the  DC  level  to  a  mimimu»».  at  the 
optimum  settings  of  the  k.  so  as  to  avoid  drift  after  the 
system  had  reached  its  near  optimum  position.   Such  an 
adjustment  is  possible  only  because  we  knew  a  priori 
what  the  optimum  settings  were.   An  adaptive  system  is, 
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of  course,  unnecessary  if  we  have  such  knowledge  of  the 
optimum  settings .   A  way  of  avoiding  this  problem  would  be 
to  heterodyne  all  signals  up  about  some  suitable  carrier 
(say  10  Kc.)  and  work  with  the  heterodyned  signals. 
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CHAPTER  IV 
CONCLUSIONS  AND  RECOMMENDATIONS 

A.  CONCLUSIONS 

We  have  shown  that  the  adjustment  procedure  does 
produce  the  desired  results  for  the  mean-square  error 
criterion.   The  time  taken  to  converge  can  be  controlled 
by  adjustment  of  the  feedback  loop  gain.   Since  conver- 
gence times  of  well  under  30  seconds  can  be  obtained  with 
this  system  and  the  system  does  perform  satisfactorily  over 
the  required  30  db  range  in  noise  power  density,  it  is 
applicable  to  use  in  an  adaptive  filter  system  for 
radio  communications.  E*urther,  it  has  been  shown  that 
the  overall  gain  in  performance  over  a  fixed  filter 
which  can  be  achieved  by  even  this  simple  example  is 
of  the  order  of  5  to  10  db  at  the  extreme  deviations  of 
the  noise  power  density  from  the  fixed  filter  design  value. 

B.  RECOMMENDATIONS 

Although  the  adjustment  procedure  worked  well  here, 
it  should  be  pointed  out  that  the  cost  of  stabilized  multi- 
plier units  might  make  such  a  system  impractical  from  an 
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economic  viewpoint  in  the  use  proposed  here.   However,  if 
use  of  this  unit  jnaterially  decreased  the  probability  of 
enemy  detection  of  a  25  million  dollar  ship,  the  expendi- 
ture of  several  thousands  of  dollars  would  be  well  justified. 
The  possibility  of  reducing  the  cost  of  operational  units 
from  that  of  the  highly  specialized  equipment  used  for 
this  experimental  work  is  great.   For  instance,  further 
investigation  of  applications  of  this  adjustment  procedure 
might  look  into  the  possibility  of  heterodyning  all  signals 
about  a  suitable  carriep  and  working  with  the  heterodyned 
signals.  This  would  avoid  the  DC  drift  problem  encountered 
here  and  may  permit  the  use  of  relatively  inexpensive 
units  for  the  multipliers  as  well  as  for  the  entire  system. 

The  design  engineer  having  access  to  existing  analog 
computing  equipment  should  consider  this  adjustment 
procedure  as  it  offers  reasonable  savings  in  filter  design 
time  over  that  required  by  present  statistical  measurement 
and  computational  techniques. 


71 


BIBLIOGRAPHY 


1.  D.  J.  Sakrison,  "An  Iterative  Method  for  the  Design  of 
Filters  for  Non-Mean-Square  Error  Criteria" ,  to  be 
published  in  IRE  TRANS,  on  INFORMATION  THEORY,  July 
1963. 

2.  S.  Sherman,  "Non-Mean-Square  Error  Criteria",  IRE 
TRANS,  on  INFORMATION  THEORY,  vol.  IT-4,  no.  3 
pp.  125-126,  Sept.  1958. 

3.  W.  Q.  Crichlow,  D.  F.  Smith,  R.  N.  Morton,  and  W.  R. 
Corliss,  Worldwide  Radio  Noise  Levels  Expected  in  the 
Frequency  Band  10  Kilocycles  to  100  Megacycles,  NBS 
circular  557, (GPO,  Wash ing ton, D. C. ,  1955). 

4.  G.  Fant,  Acoustic  Theory  of  Speech  Production  (Mouton 
&Co.,  The  Hague,  1961). 

5.  G.  C.  Newton,  Jr.,  L.  A.  Gould,  and  J.  F.  Kaiser, 
Analytical  design  of  Linear  Feedback  Controls 
(John  Wiley  &  Sons,  New  York,  1961). 

6.  D.  J.  Sakrison,  "Application  of  Stochastic  Approximation 
Methods  to  System  Optimization" ,  Sc.D.  Thesis,  M.I.T., 
May  13,  1961.  Also  available  as  M.I.T.  R.L.E.  Technical 
Report  391. 


72 


thesS608 

An  Adaptive  filter  system  for  radio 


3  2768  002  00794  0 

DUDLEY  KNOX  LIBRARY 


